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(BEST) and the CASAS. The Oral BEST is designed for adult ESL learners at the 
survival and pre- employment skills level. It uses real life materials and 
tasks to measure performance of basic language competencies. The CASAS is an 
older test, and is already well-established and accepted as a valid and 
reliable assessment tool. For reasons provided in detail, it is concluded 
that the Oral BEST test would be the better choice by the Tennessee state 
government to use in fulfillment of federal requirements. (Included in this 
document are various letters, summary sheets, and memoranda expressing 
opinions about one or both of these assessment tools.) (KFT) 
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ESOL Assessment Action Research Report 
Center for Literacy Studies 
The University of Tennessee 
March 2, 2000 

During the program year 1999-2000, seven ESOL Peer Trainers; Pat Sawyer, ESOL 
Coordinator; and Dr. Connie White, Associate Director at the Center for Literacy Studies, 
The University of Tennessee, Knoxville; took part in a program incentive grant. 

Their goal was to try out various ESOL assessment instruments, to draw conclusions from 
this research, and to assist the Center for Literacy Studies in writing a report, including 
recommendations, about ESOL assessment. This work- was suggested by the Tennessee 
Department of Labor and Workforce Development, Office of Adult Education. Hope 
Lancaster, Program Manager, provided assistance. 

As the U.S. Department of Education has mandated a uniform outcome-based reporting 
system that requires that all states collect data for programs funded under Adult 
Education, tfie assessments which determine these outcomes must be “valid and reliable”. 
They must be in the form of standardized tests or other means that meet these 
requirements in order to report -learner gains in reading, writing, speaking and listening, 
along with additional skills related to workforce assessment. 

Two assessment instruments, the Oral BEST and the CASAS were chosen for the piloting 
as they were mentioned in the NRS report as ESOL standardized tests that would be 
acceptable to determine participants’ functioning level. 

The BEST, developed by language testing professionals at the Center for Applied 
Linguistics, is a measurement tool designed for adult ESL learners at the survival and 
pre-employment skills level. The BEST uses real life materials and tasks to measure 
performance of basic language competencies. The CASAS was begun in 1980 through a 
partnership of the California Department of Education and a consortium of adult 
education agencies in California. This test has been approved and validated by the U.S. 
Departments of Education andLabor and meets the mandates of 1998 Workforce 
Investment Act. 

Current practice and theory seem to recommend using a combination of commercially 
available and program-developed alternative assessment instruments. We believe that the 
first principle in constructing an assessment system is that one should always, “do no 
harm”. Keeping that in mind, we recommend the following assessment instruments and 
practices which will give clearly stated criteria for a) entrance to placement in the 
program, b) progress within the program and c) exit to the mainstream English 
curriculum. 

We recommend that Tennessee Adult Education ESOL programs adopt the Oral Best to 
meet the NRS requirements and that the literacy section of the test also be used for 
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reading and writing assessments. (The NRS is in the process of adopting the literacy 
section of the BEST.) We also recommend that all ESOL teachers receive training in 
administering the BEST. This training can be done at the ESOL Institute, TAACE, and 
regional meetings using a module developed by the Center for Literacy Studies. 



History of the Assessment Process 

At an Adult Education Supervisor’s meeting in 1998, Tennessee Adult Education 
Supervisors requested that the Center for Literacy Studies, University of Tennessee,assist 
the state in developing an ESL assessment process through program incentive grants. 

It was decided that the work should be done with a group of ESOL practitioners in an 
incentive grant process which would identify an assessment process for ESOL 
leamers,incorporate the process into the state plan, produce training materials and work 
with the state office to provide training in the assessment process. 

At the same time, it was decided to develop an ESOL Peer Trainers’ Network made up of 
a group of exemplary ESOL instructors who were willing to lend support to other ESOL 
teachers and provide leadership andriraining regarding teaching ESOL and work-related 
basic skills. These Peer Trainers were then asked to apply for an ESOL Program Incentive 
Grant to try out assessment instruments in her/his classroom with materials provided by 
the Center for Literacy Studies. Those grant recipients were to document the results two 
times per month on the ESOL listserv, attend a meeting to draw conclusions from the 
research and assist the Center for Literacy Studies in writing a report, including 
recommendations, about ESOL assessment. 

The CASAS and the Oral BEST were selected as the tests to pilot. The first order of 
business was-to-order materials for each Peer Trainer and to arrange for training by a 
certified CASAS trainer.(To purchase CASAS material, one must first be trained in 
administering their tests.) The group then met in Nashville for a day of excellent training 
by a CASAS trainer and left the session feeling very positive about the CASAS program. 
Five Peer Trainers chose to pilot the CASAS Life Skills ESL program because they were 
interested in the fact that all CASAS assessment is linked to competencies and 
instructional materials that focus on learners’ goals. 

As previously stated, ESOL grant recipients were to document their findings two times 
per month on the listserv. The following section of this paper will be devoted to various 
excerpts from the Peer Trainers’ listserv comments. 



Peer Trailers’ Listserv Comments 

The first Peer Trainer to report on-the listserv was Ellen Bourne. Ellen is an ESOL teacher 
in Nashville, Tennessee who after teaching ESOL for five years, decided to return to 



school for her Master’s in Education. Before becoming and ESOL teacher, Ellen was a 
freelance writer for ten years. 

Ellen had chosen to pilot the CASAS and her first report was to inform everybody on the 
listserv that CASAS is not simply a test for placement. “. . but a system to assess* place, 
instruct, and tract students.” She continued by writing that “CASAS also provides 
accountability for the funding sources about how well teachers are teaching and how well 
students are learning with its pre and post tests.” 



After explaining the CASAS’s testing process, Ellen wrote a few of her own observations. 
1 . The test is culturally biased, at least in the use of names, none of which indicated any 
other nationality (Ed Smith, Ronnie Smith, etc.). 

2. 1 tried to test all of the students at once, but of course, this was impossible. (Even 
though I called the students to remind them to be sure to be there, some didn’t show on 
the testing day.) 

3. Since the test must be proctored, it really messes up your teaching schedule (1 and 1/2 
hour classes) to be testing one set of students for such a long time when the others have 
already taken the test. 

4. Some students were mildly annoyed that we took this kind of time on the tests even 

though I explained the need and use of the results. 

\ 

5. At one site where I teach, we have the luxury of a ‘^tester’’ who tests and-brings the 
student to the proper level.- Ican-t imagine teachers-being able to spend this time with 
every student in an open enrollment situation without a “tester”. 

A few days later. Heather Nieely-anESOL teacher-in the Kingsport, Tennessee adult 
education program sent a report via the fistserv. Heather-is an experienced ESOL teacher 
who has worked as a teacher and also^as an Adult Education Supervisor. She discussed 
her program and how she was progressing in her assessment piloting program. She also 
noted that “Yes, this testing-business is-cumbersome,- but will be required soon to 
document via tests the levels of our students and the progress they make. 



Again, a comment was made concerning the time assessment takes in our ESOL classes. 
Heather wrote. “Let’s put our heads together to^come op with a solution that may free 
trained ESOL teachers fWe are a rare breed! !) to do what they do best-TEACH.” 
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From the beginning of this action-research project, the problem of how to test an 
ESOL classwithout losing ourstudents was a common theme. One of ourPeer 
Trainers said that the third time she tested, shehad-only three of theoriginalgroup 
of students who^ had started with-her; 

As all adult ESOL teachers know, -our-students have a limited amount of time to 
spend in class and they expect that time to be worthwhile. 

The next concern that was addressed was CASAS’s use of the “bubble sheet”. 

Sandra Fugate, an experiencedESOL Peer Trainerwhoteaehes in Claiborne County 
wrote the following. “I attempted to test two of my students with the CASAS -Appraisal. 
They had never seen a bubble sheet. The appraisal said-that-if they had trouble filling-out - 
items 1-5, to discontinue testing and try assessing with another form. I pursued a couple 
of pages-of the-Listening test. They-eoukl not-keep-up-with-the tape. These two-students- 
went to the dth grade in Mexico. They were discouraged and so was I. ” I am undecided 
about CASAS at this point.” 

After receiving this message, Ellen replied, -T suggest that you spend a considerable time 
preparing levels below advaneedto-take this type-of test-. Most of my intermediate 
students havemet been to college and have no facility with-filling out a bubble sheet. 
Because the students were upset with the testing process, I gave them a page with lots of 
different facial expressions on it and asked them to-circle the way they felt when they took 
the test. We talked about how frightened we^cambecome when we take a“test” but I 
reminded them that they all took a test when they wereglaced in the program and this was 
a similar test, etc We also talked-about how we don’t like change, but a change in the 
type of test we take is coming next fall. Teachers and students alike will have to 
change. .. anyway, I felt better about it all after we talked as a class.” 

Another common theme running through these reports about the CASAS was how 
uncomfortable the ESOL students were when taking the tests. 

Diane Cohn, a Peer Trainer from Franklin, Tennessee, who teaches ESOL and is also 
involved in a pilot program with-McDonalds restaurants wrote, ‘Tor the writing part of 
the assessment, most students got so nervous listening to it that they didn’t know what 
they were supposed to write. When they were being tested on the reading part of the test, 
those that have had very few years of formal schooling or none at all felt very depressed 
and inadequate. Several students just quit listening and put their pencils down. Our fear is 
that students will feel so badly about themselves after an assessment that they will give up 
and not return to class. 

One Peer Trainer, Anne Pittman, from Memphis, TN, was able to select 10 students 
and isolate this group from the rest of the ESOL students for testing purposes. 
Anne’s experience was positive. Her comments are as follows. 
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“The students who have been in my class are actually rather excited by the process and 
feel that they are participating in a bit of Tennessee history. All in all the students and I 
liked the CASAS very much and (once we got the “door closed” on the incredible 
busyness of our Memphis program) are really enjoying the process of our assessment 
pioneering. I also want to comment on how easy these tests were to grade. For those 
who haven’t seen the CASAS assessment’s grading sheets, they come in a sealed format 
such as those that come m the mail with a notice that one’s Tennessee license plate 
renewal is due. The students respond on the outside and then the perforations are tom 
open and their bubbled responses are immediately revealed to be correct or not. It is my all 
time favorite grader-friendly format.” 



At a later time, Anne’s response was, “My group is now wearying of the testing 
process. In spite of great promises of dedication through March 2, my attendance 
was way down last night. This, of course, could have been serendipitous, but I don’t 
think so.” 



* ****** 



Along with our own Tennessee Listserv, we have been monitoring the NIFL listserv 
and talking with teachers and supervisors from other states. One comment from a 
Special Populations Coordinator from another state was, “...our teachers just got 
training on the CASAS in January so I don’t have a lot of feedback yet, In training, 
they were not too enthusiastic about the bubble sheets (thought it might be hard to 
train ESL students to use them efficiently). Personally, I think the “oral interview” 
and the “writing” (dictation) pieces on the CASAS are a joke. The listening test is 
really a reading and listening test. 

The Peer Trainers who were piloting the CASAS assessments felt that they were 
ready to complete the project by the time we all met March 2, 2000 in Nashville. 
None of them, at this time, knew what final decision our group would make because 
even though they had been communicating about the assessment project on the 
listserv, they were waiting to hear from the two Peer Trainers who had piloted the 
BEST. 

Cindy Barnett, our ESOL Peer Trainer from Henry County, Tennessee, was the first to 
report on her experiences with the BEST. In several of Cindy’ s listserv messages and at 
our meeting in Nashville, Cindy emphasized the importance of training for the person who 
would be administering the BEST. Some of Cindy’s comments are as follows. 

1. “The BEST manual emphasizes the importance of the examiners becoming thoroughly 
familiar with the procedures and scoring rules* especially for the oral section It is 
recommended that one practice giving the test to colleagues or volunteer students. The 
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more you use this test, the more comfortable you become with-it.” (The Center for 
Applied Linguistics distributes a training video which was developed by Clackamas 
Community College in Oregon City, Oregon. This training video can be used with 
the BEST Oral Interview Short Form.) 

2. My overall impression after giving the BEST to 10 students is that it’s a very good test. 
Once you become familiar with the questions and the scoring, it’s not too hard or 
time-consuming to give. 

3. There are two forms of BEST available, B and C so you would use one for the pre-and 
one for the post-test. 

i 

4. The BEST would be excellent for placing new students. It can also be used to 
provide diagnostic feedback. For example, my student who didn’t score well on the 
handling money questions was proficient at telling time. Many others had some 
trouble following directions on a map. 

5. Many of my students were smiling as they finished the test and said it wasn’t as 
hard as they thought it would be and they felt they had done well. The student who 
really gushed about it happened to be the one who scored the lowest. The Oral 
BEST seems to make the students feel good about what they CAN do and doesn’t 
focus on what they can’t.” 



Connie Mayes, an experienced ESOL Peer Trainer from Sevier County agreed with Cindy. 
Connie wro(e the following in her listserv reports. 

“I too think the -BEST tests what we need to know ^nd it only takes about 15 
minutes. Students don’t seem to mind it. I have also given this test to some of my 
advanced students and have seen interesting results. Since correct grammar is 
essential to getting the top score, I have been finding deficient areas in some 
students. It has also been an eye-opener for them. 

AH in all, I am pleased with it. I make notes as I give the test and use that 
information -with the scores so I can determine what I need to teach. So far in my 
testing, the student’s scores match my observations of my students. I ask new 
students to come in for a private interview ahead of their first class so I can evaluate 
them. 

I like it because it does not disrupt my class and it is SHORT.” 
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Final phase of the ESOL Action Research Project 

On March 2, 2000, we met once again in Nashville, TN to write our report, including 
recommendations, about ESOL assessment Each Peer Trainer had prepared a written 
report of her research (see Appendix B) which she read to the group. After the reports 
were read, questions and answers were given and then the pros and cons of each test were 
written on large sheets of paper which were placed on the walls of the meeting room. 
These comments were read and studied by everyone. They are as follows. 



CASAS Pros 

Good customer service 

cross-referenced materials on what to use once students are assessed 

class profile is helpful for instructor’s planning 

a well organized system that works for teachers and administrators 

the levels are useful for accountability 

accurate for literacy (reading) 

adequately tests listening and reading 

the manual gives LOTS of information 



CASAS Cons 

does not test writing and product 

time, time, time— takes too much time to administer 

too costly 

may intimidate or chase off students 
leaves us with a feeling of “surely there is something better” 
may not be user friendly (difficult for students to take) 
Caucasian cultural bias 



BEST Pros 

relatively inexpensive 
not intimidating to students 
students feel good about taking it 
easy to score 

short, oral and literacy together is 1 hour and 15 min. 
literacy test can be taken on own 
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literacy test is short and tests life skills 
has a grammar section 
students can see relevance of test 
is even helpful for non-readers 
was developed for ESOL students 

BEST Cons 

can not discriminate among highest levels 
has to be given one-on-one 

must be teacher administered, not appropriate for volunteers to give unless a volunteer is 
highly trained 

some questions about BEST’S definition of “fluency” 

scoring is subjective unless teacher is thoroughly familiar with the test. 

During our discussions of these pros and cons, Diane Cohn mentioned that she had been 
giving the literacy section of the BEST to her students. She was very enthusiastic about 
this experience and had many positive things to say about the test. She had chosen to pilot 
the CASAS and was, we thought, in the enviable position of being able to compare the 
two instruments. 



The Final Decision 

After reading reports from other states, discussing the pros and cons of our own research, 
and then again remembering our first principle to “first do no harm”, we decided to 
recommend the Basic English Skills Test (BEST) as the most appropriate assessment for 
adult ESOL programs in Tennessee. 

The BEST will work as an excellent test for ESOL students at all 6 levels. It will NOT be 
effective though, as an exit test at the ESOL Advanced Level. Therefore, we are 
recommending the following test to be used as an exit test for the ESOL Advanced Level. 

TABE (Tests of Adult Basic Education) 

Level M, Form 7 and 8 

Reading Test. (See Appendix D for address and information.) 



Final Comment 

My thanks to the excellent work done by the ESOL Peer Trainers of Tennessee and the 
wonderful assistance of Dr. Connie White at the Center for Literacy Studies, The 
University of Tennessee and Hope Lancaster, Project Manager, Tennessee Department of 
Labor and Workforce Development, Office of Adult Education. 

We join together in wishing our ESOL adult education teachers success in testing. 

Pat Sawyer 

Center for Literacy Studies 
The University of Tennessee 
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APPENDIX A 



START UP KIT (with training video) IS S200.00 



Basic English Skills Test 
Center for Applied Linguistics 
4646 40th Street NW 
Washington DC 20016-1859 



contact: 

Laurel Winston 
Tel 202-362-0700 
FAX 202-362-3740 
e-mail laurel@cal.org 



TRAINING FOB TENNESSEE ESOI TEACHEBS IN THE BSE OF THE BEST WIU. BE 
GIVEN AT THE ESOL SBNNEB INSTITBTE ON IBNE 20, 2000. 



APPENDIX B 



PEER TRAINERS’ REPORTS 
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Testing the CASAS 
Ellen Bourne 



Program tested: Non-intensive ESOL classes, day classes two days 

a week, three hours of instruction per day (two teachers, 1 1/2 
hours per teacher) ; non-defined curriculum. Students drawn 
mostly from the Vanderbilt University area, Nashville; very few 
are employed; conversation/communication the traditional goal of 
instruction; open enrollment 

Stated purpose: 

assessment, instruction, accountability (life skills) through use 
of appraisal, pre and post tests; program must teach skills that 
are tested if to be used for accountability (depends on 
program/test fit, in other words) ; originally designed for ABE 
students 

Numbers and levels tested: 

7 intermediate students took the ESL Appraisal Form #20 . 
(Languages represented = Ukrainian, Spanish, Indonesian, 

Polish, Russian, Chinese, ) Four others took either the 
listening, or the reading appraisal tests 

6 advanced students took the ESL Appraisal Form #20 and were 
pretested on Form #37 . (Languages represented = Turkish, Korean 
and Chinese.) Seven other students took all or parts of the 
appraisal test, (they weren't given on the same day.) but no 
pretest (Languages represented = Korean, Japanese, Italian.) 

Students took the listening and reading and writing appraisal 
tests. I did not administer the oral test. 



Admini s t r a t ion : 

Appraisal 

Listening 25 minutes 

Reading 30 minutes 

Writing 5 minutes 



Administration: 

Pretest 

reading approximately 60 minutes 



Cultural Issues: 

Content /Format 
Language bias : 



Task bias : 



pictures universal 

some bias in that no students referred to in 
the. articles for testing have non-Anglo 
names; vocabulary pretty stiff for an 
appraisal for all levels: convenience, 

combination, hazardous, released, irritant, 
internal . _ _ , • _ 

mass of personal identification information 
required on appraisal sheet overwhelming for 
intermediates; confusing to have other 
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information on the sheet that students do not 
fill out (“placement into program”; "Agency"; 
"Class".; etc.); use of bubble answer sheet a 
new concept and difficult concept for 
intermediates; Use of medical form that they 
do not fill out but rather answer questions 
about confusing for some. 



Range/breadth of content: 

Age adult 

Language type basic skills; life skills, employability 

Language domains broad coverage and depth 

Language skills four basic skills: reading, writing, 

speaking, listening 



Documentation/manuals : 

Interpretive wealth of information available from the 

training session. All materials listed 
below were obtained from the training 
session. 

skill level descriptors for ABE; raw to 
scaled score conversion tables,* competency 
coding system; competency lists; thorough 
administration information; individual 
student profiles showing test 
content by item; instructional materials 
guide (listing textbook and publisher) and 
more. 



Comments: If we are to use this assessment we must take into 
account that needs and goals for different sites, times and 
locations around the state* are different. One site where I 
caters to wives and husbands of Vanderbilt postgrad students who 
y^-'-l never work in this country*. The advanced levels are quite 
lgh and these students could easily take graduate work (many are 
professionals, doctors, lawyers, etc.), but opt for our free 
program instead of paying tuition at the universities. These 
students have no trouble with the CASAS. 

But for other levels below advanced or high intermediate 
teachers would have to teach test-taking techniques: how to 

prepare emotionally for test taking; how to follow directions; 
how to use bubble answer sheets; hew to take a test without using 
^ ®^ct ionary ; how to read for content, i.e. most important 
mfcrmation/sentence in an article. 

I also think that we will begin teaching to the test, rather 
v-han the other needs students may have, if we are to post- tes** 
with the same test. 

I think this is a fair and accurate test for advanced level 
students and perhaps for the beginning levels with their mostly 
pictorial type of test, but the middle levels seem to fall 
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through the cracks in terms of the issues above and in terms of 
vocabulary for the appraisal test . 

Recommendations: The test is too time consuming. Perhaps 

volunteers could be trained to test students/ however/ -since it 
is difficult to predict when new students will appear, this 
method of test administration might prove hard to manage. 

'Pgachgrs could be trained to administer the test, but most o 
them are too busy teaching to spend a lot of time assessing 
(unless, of course, they are convinced that the test contains all 
the learning they want for their students) . Trying to corral 
students for this lengthy test while the rest of the class is in 

session seems counterproductive. 

Perhaps we could use the BEST for lower levels and break out 
the CASAS for the upper levels.. Alternatively, perhaps neither 
instrument should be used for placement but rather as a starting 
point for progress after the class is established and as a post 

test . § „ 

Teachers need to be f amiXiarized with the test and its 

contents and with the process for answering (use of answering 
sheet). The emphasis must be changed from a communications (oral) 
competency to a literacy competency. 



Scoring: 

Of 11 advanced students who took the appraisal all had raw scores 
of 17 or higher (out of 20 questions) that translated to a scaled 
score of 227. This put them into pretest 37 and 38 post test. 
Scores on the pretest ranged from 25 to 38 out of 40 questions. 

Of 8 intermediate students wha took the appraisal, raw scores 
ranged from 13 to 3 (out of 20 questions) . Four students had a 
raw score of 13 (scaled score = 218 = form 35); ^student s tudent 
raw score of 12 (scaled score of 216 = form 35, ® ls ^L n 1 3 ®^ d J nt 
had a raw score of 10 and a scaled score of 211 (= form 33) ,i 
student had a raw score of 8 and a scaled score - score Q f 

33 also); 1 student had a raw score of 3 and a scaled score of 
194 (= form 31). This class had wide discrepancies in the 
listening portion of the appraisal instrument as well. . . 
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CASAS Assessment 
Diane Cohn 
Williamson County 



To evaluate the CASAS Assessment I tried to look at the pros and cons of the instrument itself. 
What is so wonderful about CASAS is the fact that t here are so many levels. A beginning 
student in particular can show a lot of progress, which is good in the age of accountability. I 
really liked the literacy level tests, forms 27 and 28. All of the students who took that test felt 
good about what they knew. This is so crucial at this level and I don’t think we’ve really had a 
good instrument at this level before. Another good factor is the Appraisal answer sheets are very 
easy to grade. Any time saving factor is truly appreciated by the teacher. Perhaps the best factor 
about the CASAS it the fact that it tests R eading, Writing, and Oral comprehensi on. This is 
crucial in trying to understand our ESL students and gives us a better over-view of their abilities. 
The CASAS also is so organized in helping the teacher know exactly what skills the student 
d oesn’t know an d then has all these skills cross-refer enced with teaching materi als. That is a 
huge plus. 

St. Thomas Hospital literacy classes in Nashville use the CASAS in their workplace assessment. 
They use the CASAS Workforce Learning Systems Appraisal in Reading/Math to determine if 
an employee is at the required levels of his/her job. If the employee does not make the required 
score, set by both the teacher and supervisor, they are required by management to enroll in the 
program for a minimum of 8 hours a month. They then retest after 12 hours of class time. The 
teachers in this program like the CASAS assessment because it is skill specific. They then can 
teach the necessary skills in the class that relate to the student’s job. 

Overall, I don’t think that there is another test on the market that has so many levels, and at the 
same time, have all of the skills cross-referenced for the teacher. Both of these facts are crucial 
for our ESL teachers. 

The Cons about the CASAS can be summarized in one word — TIME. A teacher has to give two 
tests to each student, one an appraisal and then follow with a reading test before the student’s 
level can be determined. Each test takes at least an hour and a half to administer, taking into 
account filling out the forms and reading the directions. This is very time consuming, 
considering most ESL teachers teach one or two classes a week. This takes a whole week of 
their instructional time for assessment. 

Another big problem with the CASAS for me was the administration of the test. I have had 
CASAS training, I read the manual at least three times and I still felt insecure about 
administering the test. How would the average part-time teacher feel who teaches one or two 
classes a week? I have several questions that I hope my esteemed colleagues can answer. 

1) Do I use the reading appraisal score to determine the reading test form? If so, where 
in the manual tells me that? 

2) How do I use the listening score in my assessment? 

3) Do I give the same reading test form again as a posttest? Again, where in the manual 
does it tell me that? 
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4) How do I use the class profile? 
any explanations in the manual. 



don t understand all of the symbols and I can't find 



In one instance in particular 1 question the results of the CASAS. Student A took test form 3 I 
received a — 0 score, which put her into Intermediate Basic Skills level. Student B took test 
torm j3 received a 21 0 score, which put her into Beginning Basic Skills. Student B functions at 
a higher level in class than Student A and placed higher on the Appraisal, yet her assessment is 
lower than Student A is. This doesn't make sense to me. 

Another drawback to me about the CASAS is the recommended 60 teaching hours before one / 

can post test. I feel this would be a nightmare in bookwork, and one would be testinu and re- 
testing continuously. 

The bottom line with the CASAS is-IS CASAS USER FRIENDLY? Can a teacher use it 
efficiently and find what they need to know? I keep thinking about the majority of part-time 
SL teachers who teach one class at night. How would they feel about the time involved in 
giving the CASAS? Would they feel qualified to give it? I haven’t even touched on the cost of 
the training. Would districts be willing to pay for CASAS training in a field that has a majority 
of part-time teachers that come and go? And is the CASAS an efficient assessment in a field 
where the teacher never has enough time? Does the assessment take away too much 
instructional time? Can we get the similar results by using another assessment? 

VVhat have I learned from using the CASAS? I have learned that giving this test takes a lot of 
class time and preparation time. Time I was happy to give, but I’m not sure the average teacher 
has to give I learned that in reading how to give the test— the more I read the more confiised I 
got. It took me several readings of the manual each time before I actually administered the test 
I learned that I had to give up almost of whole week of instructional time to administer this test 
to my advanced group. This group only meets two days a week. I learned that when my 
students come to my class I want them to walk out feeling successful. I’m not sure that taking 
the CASAS had that effect on them. Finally, I learned that I want one instrument to assess my 
students when they come to class that doesn’t take so much time. I’m not sure I can say that 
about the CASAS Assessment. As you can see, I have asked more questions than I have 
answered which sums up my feelings about the CASAS, Ambiguity. 
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STUDENT RESPONSES AFTER TAKING THE CASAS APPRAISAL 

I felt this test is good for me because I think 1 need to get used to listen to different kinds of 
person, voices and speed. I hope that tape voice more clearly. 1 can easy listen. 

1 think that this test is good, very good at everybody because is much practice and I’m wanted 
that every months ends a test to be better. I think I need more practice because my test is not 
very good but I will try to studied more. 

I like the tests because I need to know about who many learned and if I go well in my study. 

1 like the test because for me it was so hard and is very important for know how is my label and 
what can 1 do for better my English. 

1 like the tess but I things is a little hard for me when I was listen. 

I think the test is good but Part two for me was unclear or was more difficult to understand what 
really wanted. 

I felt it was difficult for me especially listening. This test let me know that I need to study more 
English. Thank you so much. 

It’s make me crazy. It’s tape just like a telephone call. For foreigners sometime understand not 
only what they say but what they do (action and face). So, this is good practice, but it’s hard. 

1 had a headache. 

I think that test is needed for us (me?) twice a year. So we can know how much we learned and 
the parts we have to improve. 

I think it wasn’t a very hard test. It was a sometimes to slow for easy questions. It could go 
faster. I think it is good to do those test to control how much the students are learning, and it 
didn’t made me feel bad. I think you should keep going on such tests. 

I like test. 

I think it is a best thing what we take a test. Because I could know how I understand my English 
ability. But this test is a little hard for me. I like to take a test. 
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REPORT ON CASAS ASSESSMENT Heather Nicely 



6 March 2000 






GENERAL COMMENTS: 

The students who took the CASAS in the Kingsport City Schools Adult Education program were mostly 
adults; all were enrolled in the Adult ESOL class(es) in the Adult Basic Education program. Those taking 
the listening portion of the ESL Appraisal, Form 20, took it as a group, either at Lee School, Dobyns- 
Bennett High School, or the Kingsport Even Start program. The reading portion of the ESL Appraisal, 
Form 20, was mostly group administered as well. The next level of test, the reading test on Levels A-D 
was administered once as a group and the remainder of the time, one at a time, as the students reappeared 
in the ESOL class. 

Things I noticed about the test included that the dictation portion was something the student either could 
handle or could not. Only a few students just wrote down a word or two; most either got most to all of the 
words or could write nothing. I did not administer the oral portion of the test, but looked it over, and 
came to the conclusion that it might reveal things about the English ability of a student to an experienced 
teacher, but would not be of much help to a beginning teacher. Most students took longer than one hour 
to complete the second reading test, even the fairly fluent students. That may have been a factor of their 
scores on the ESL Appraisal test qualifying them to take a fairly difficult next test. When we did the 
practice items as a group, even though I pointed out where the practice boxes were, many students marked 
the practice items, not in the correct box, but in the spaces for questions 1 and 2. There was a marked 
similarity in the scaled scores on the reading portion of the ESL Appraisal and the reading test, levels A- 
D. (See below for a list of names and scores.) 

Student comments included wondering why a test was being given before there had been many lessons 
(that from a newly enrolled student.) Some said the reading portion was fairly difficult. I thanked them 
profusely for being "guinea pigs" for the state of Tennessee as we tried to ascertain which test would best 
be used for accountability in the adult education program. 

A test of this type would be a burden for a single ESOL teacher handling all the needs of students from 
other countries. Testing would occupy a great deal of time and take away from teaching time. In the 
evening class situation in my program, I would have to depend on our regular testing person to do the 
ESOL testing so that I would be able to concentrate on the lessons in English that my students need and 
want. 
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TEST RESULTS FROM KINGSPORT CITY SCHOOLS ADULT EDUCATION ESOL PROGRAM: 



Student name 


ESL Appraisal Reading Score 


Reading Test Score 


Ursula Bonaldi* 


233 




241 


(C) 


Kyoung Ae Tate 


228 


235 


(C) 




Noella 


213 




223 


(C) 


Marilena Freitas 


218 




221 


(C) 


Antje Guempel 


227 




234 


(C) 


Jin Chul Lee 


233 




243 


(D) 


Ronaldo Godinho 


233 




258 


(D) 


Jovana Pivac*** 


230 




231 


(D) 


Edmar Jaremcivc 


228 




243 


(D) 


Soon Park 


190 




202 


(A) 


Milan Pivac 


211 




196 


(A) 


Kyoung Moon Yang** 


206 




231 


(B) 


Asim Brka 


204 




210 


(B) 


Suvada Brka 


201 




209 


(B) 


Kidjana 


204 




203 


(B) 


Ji Young Lee*** 


209 




201 


(B) 


Mladen Pivac*** 


204 




203 


(B) 


Enrique Franco 


162 




191+ Pre-A 


Lidia Villalpondo 


209 




194 


(A) 


Patricia Garcia 


198 




196 


(A) 


Rosalba Blancas 


198 




220 


(B) 


Maria Montes 


211 




215 (C) 


Esmeralda Franco 


216 




214 (C) 


Alicia Franco 


224 




215 (C) 


Francoise Amboko 


230 




232 (D) 



♦Ursula is a be ginnin g student in the morning class at Lee School. However, she already knows 4 other 
languages and is catching on to English rapidly. Her oral fluency is not nearly at the level of her ability to 
read in English. 

♦♦Kyoung Moon was using his electronic translation dictionary (Korean) at the beginning of the second 
reading test. When I became aware of that, I made him put it away. Hence, his somewhat skewed score. 

♦♦♦Younger students. Ji Young is 11; Jovana is 18; Mladen is 15. 
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Results of Action Research on the Oral BEST 



The Basic English Skills Test (BEST) is a standardized, criterion-referenced test designed 
to test listening comprehension, speaking, reading, and writing skills at an elementary level. It is 
intended for use with limited-English-speaking adults for whom information on the attainment of 
basic functional language skills is needed. It was originally developed in 1982 as a part of the 
Office of Refuge Resettlement Mainstream English Language Training (MELT) Project. Two 
forms of the BEST are currently available from the Center for Applied Linguistics. The test 
results are reported in Student Performance Levels (SPLs). The BEST is designed to provide 
information for purposes of placement, progress, diagnostic, screening, and program evaluation. 

The BEST consists of two sections: the Oral Interview Section and the Literacy Skills 
Section. I have not seen the Literacy Section so I can only report on my experience with the Oral 
Section. 

The Oral BEST is an individually administered, face-to-face interview that includes a 
series of simulated real-life listening comprehension and speaking tasks, such as telling time, 
asking for and following directions, counting money to pay for items and verify change, and 
conversing socially at an elementary level. A reading task (completion of a short biographical data 
form) serves as a screening device to identify students for whom the Literacy Section rpay be 
appropriate. It takes about 1 5 minutes to administer. 

The examiner must become thoroughly familiar with the procedures and scoring rules 
before administering the Oral BEST. It’s also a good idea to practice giving it to colleagues or 
student volunteers. You must follow the instructions exactly. The specific language of tlje test 
questions to be spoken aloud by the examiner is printed in boldface. You may repeat each 
question only once if the student doesn’t understand or asks for repetition or clarification. You 
may not rephrase the question. 

The Oral BEST tests listening comprehension, communication (comprehensibly and 
grammatically correct responses), and fluency. These scores are added to give the total score. It’s 
easy to add the scores. The total is a scaled score, which is then correlated with the SPLs using 
the correlation chart included in the manual. An overall pronunciation score is given separately. 

In using the Oral BEST to measure progress, one of the two versions available could be 
used for the pre-test and the other for the post-test. The manual recommends at least six to eight 
weeks between tests. I would probably wait 16 weeks, since my classes only meet two hours 
twice a week and 16 weeks would be the natural division of the school year. 

I administered the Oral BEST to eleven students and soon began to feel comfortable with 
it. I feel it’s a very good test and was made just for students like mine. It is easy to administer, it 
takes only 1 5 minutes per student. It assesses basic functional language skills. Since it is 
performanced-based, it can easily be tied to curriculum and instruction. 

It was not an intimidating test for my students, who are in the intermediate and upper- 
beginning levels of ESOL. In fact, I think it made the students feel good about what they were 
able to do. My student who scored at the lowest SPL Level, Level 1, came away from the test all 
smiles saying how easy it was and how well she felt she had done on it. I think it’s important that 
our tests not make our students feel inadequate about themselves or their abilities. The 
administration/scoring instructions for the Oral BEST say to make the student feel as comfortable 
as possible by maintaining a pleasant manner and tone of voice. “Every effort should be made to 
give the examinee a positive attitude towards the test. Once the test begins, the examiner should 
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be careful not to give any indication to the examinee as to whether an answer is right or wrong; all 
responses should be accepted with a smile or nod. Remember that this is a testing and not a 
teaching situation.” 

I have not been able to administer the test to my migrant farm worker students, who are in 
Mexico now. They are at the literacy and beginning levels. I think the Oral BEST would be quite 
appropriate for them. I was interested to read Heide Spruck Wrigley’s, Sr. Research Associate ofr 
Aguirre International, entry on the NIFL-listserv, which Pat Sawyer forwarded to us on Feb. 18. 
She said that it can be difficult to assess progress for beginning ESOL students. She suggested 
choosing an assessment instrument that “captures how well students are able to communicate in 
spoken English (verbal proficiency).” She said the first component of an assessment framework 
should be oral proficiency assessments for spoken English for ESL. “The BEST seems to work 
for many programs.” 

I think it would also work for ours and I recommend it for the Tennessee adult ESOL 
programs. 



Cindy Barnett, ESOL teacher 
Henry County Adult Education 
March 1, 2000 



RESULTS OF BEST TEST RESEARCH 

—Connie Mayes 

4 

The BEST oral test is a short test which gives a score for listening comprehension, 
communication, fluency, and an overall student performance level. It is easy to give 
and takes about 15 minutes. Students do not seem to mind taking the test. 

Pros of the BEST: 

1. Easy to give. 

2. Easy to score. 

3. Short test. 

4. Results in useful information. 

5. The scores are useful for placement of student. 

6. Can be used for all levels of students. 

7. Easy to show gains. 

Cons of the BEST: 

1 . Tester must almost memorize the test. 

2. Pictures are very dark. 

3. Must be given one-on-one. 

4. Recommendation of post-testing in 6 weeks. 

I don’t think there are a lot of cons for this test. For ESOL we need something short 
and simple that gets the job done. The BEST is easy and to the point. Our students 
often don’t know what or why we are doing something, and they don’t understand 
when we try to explain. If we spend a lot of time testing, they will simply not return 
to class. The recommendation of post-testing at 6-8 weeks is impractical for me, as 
I would be testing all the time and not teaching, and I think the students would quit 
coming if I do too much testing. 

I have fomid it fairly easy to use the BEST, as I test the students when they come 
for their pre-class interview. Some students show up for class unannounced, and 
then I try to get them after class. Since the test is short, this is fairly easily 
accomplished. 

I have tested new students and long-time students. For the most part, the results 
have been what I expected. Something interesting, though, is that a student who 
cannot read and write is able to get a high score on this test, because it is an oral 
test. The teacher might not want to use the test score for placement of this student. 

I don’t consider this a problem. 
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RESULTS OF BEST TEST RESEARCH 

--Connie Mayes 

Based upon my experience, I recommend that the state adopt the BEST test for 
their reporting purposes. We can easily give them the scores they want, and it 
should be easy to show gains with this test, as all these skills are part of our 
curriculum. 
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APPENDIX C 



Heide Spruck Wrigley 
Senior Research Associate 
Aguirre International 

“Just a couple of words of caution. If you want to pick up gains for beginning ESL 
students, you really will need to select an assessment that captures how well students are 
able to communicate in spoken English (verbal proficiency). So that will mean at least 
two assessments (one for verbal proficiency and another for picking up the changes in 
literacy skills... 

In addition, if you have folks at the very lowest level, the gains they make may not show 
up on a standardized test so it can easily look like they have not learned much of anything 
when indeed they have. 

My suggestion would be to develop an assessment framework that minimally combines the 
following: 

1 . For ESL: Oral proficiency assessments for spoken English for ESL (the BEST seems 
to work for many programs). 

2. For first level literacy learners: an alternative assessment for students new to literacy 
that combines a demonstration of what students can do (reading and writing portfolios), 
student self-assessments (can-do lists, for example); and teacher assessments. 

You could supplement that with a test that assesses reading “subskills”. 

Carol H. Van Duyzer 

National Clearinghouse for ESL Literacy Education 

(In response to a question asking if the TABE could be used for ESOL students at the 
advanced level.) 

“As far as the TABE is concerned, I think you are right to suggest that it be given to 
students who test at SPL 6 and above or top out of the BEST oral or the BEST literacy. 
Texas gives the BEST oral. If students score less than 5 on the reading and writing 
screening portion, they are not given the BEST literacy. Those with scores higher than 5 
on the screening also take the BEST literacy. Those who top out on the BEST oral are 
usually moved over to the TABE.” 



APPENDIX D 



TABE (Tests of Adult Basic Education) 
Level M, Form 7 and 8 
Reading Test 



This test was normed in 1993 and will test levels of accomplishment for 
students working at the 3.6 to 6.9 grade level. 



CTB/McGraw-Hill 
Customer Services 
20 Ryan Ranch Road 
Monterey, CA 93940-5703 



The Consultant for Tennessee is 

Tom Cannon 
(615) 855-3662 
tcannon@ctb. com 

or 

You may call the Atlanta, GA office at 
770-622-4300 
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